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Abstract 

This study looked at the instructional and assessment effects of using learning journals in 
three distance asynchronous computer conferencing courses (n=18, n=16, n=17). The 
instructor used a design-research methodology: each iteration of the course involved 
modifications to how learning journals were used based on analyses of the responses 
and results from the preceding course. Modifications included: a) use of orienting 
questions; b) question content, c) journal assessment and d) amount of scaffolding. 
Protocols were analyzed with a view to characterizing students’ epistemic cognition from 
two perspectives: belief mode (rationalist epistemology, self analysis, norms of inquiry to 
defend competing beliefs) and design mode (knowledge building epistemology, collective 
responsibility, norms of inquiry to support idea improvement and explanatory coherence). 
Changes in metacognitive reflection and learning journal activity were related to 
measures of learning. As a pedagogical tool, learning journals with directed questions 
(scaffolding) encouraged self-awareness of learning and epistemological reflection. 

Theoretical Background 

Journaling offers a context for making internal mental dialogue explicit. It involves self- 
analysis and reflection on events, discussions and ideas. This process can be structured 
or free flowing, individual or shared, and can often deal with metacognitive issues as well 
as content (Andrusyszyn & Davie, 1997). Kerka (1996) believes that journaling can 
“provide tangible evidence of mental processes” (p. 3). Journals can aid in the ‘making of 
meaning’ through allowing the connection of past to current experiences. Because 
journaling is a combination of writing and reflection, it is a tool that is suited for both 
instructional and assessment purposes in online learning environments (Hansen, 2005). 
Connell (2000) examined two graduate courses in which learning journals were used. 
Students reported that learning journals promoted a deeper level of understanding of both 
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the course content and of their own thinking and understanding. This deeper level has 
also been observed in student online journals used in other computer conferencing 
environments (e.g., Heflich & Putney, 2001). 

Journaling involves self-analysis and reflection on events or discussions (Andrusyszyn & 
Davie, 1997). Self-analysis is a significant focus in graduate studies because an 
important part of graduate learning involves the development of epistemic cognition. 
Research on epistemic cognition has shown a relationship between epistemic beliefs and 
academic performance (Schommer & Walker, 1995); and between a rationalist 
epistemology and skill in argumentation (Kuhn, 1991). 

We take the position that an important part of graduate learning involves the development 
of a rationalist epistemology. Such a perspective has been shown to proceed 
developmentally from earlier objectivist conceptions, to subjectivist and finally to 
rationalist conceptions of cognition, although the extent of individual trajectories and 
timing vary widely (Baxter Magolda, 1999; Kuhn, Cheney, & Weinstock, 2000; Perry, 
1970). An important, but less examined, part of schooling adds the requirement of 
community-level cognition that is essential to collective cognitive responsibility for 
community knowledge, a central tenet of knowledge building. Knowledge building also 
involves enculturation of students in the process of knowledge creation and work in 
design mode. For it is in design mode that ideas are examined for the purpose of 
improving their explanatory power and usefulness, asking questions such as: What is this 
idea good for? What does it do and fail to do? How could this idea be improved? (Bereiter 
& Scardamalia, 2003). From a design-mode perspective problems in understanding can 
serve as guides to conceptual breakthroughs. 

Rationalist epistemology identifies justifiable norms of inquiry that enable decisions to be 
made between competing beliefs and positions. As one student commented: 

For me it is now perfectiy dear why we have iearning journais and I think that they 
shouid be an integrai part of any graduate /eve/ course. Here is our piace to “make 
sense of things" to investigate our own iearning and see where the hoies He. In my 
eariy confused weeks it heiped a iot to write down the various things I couidn’t 
understand and why I couidn’t understand them. It also helped to write my own 
metacognitive thoughts because once they are down in words it was a way of tackiing 
r/?em.(Jane, Course 3) 

Another significant goal of graduate learning is developing an “innovative voice” — to 
become an independent researcher (e.g., Baxter Magolda, 2001) capable of generating, 
understanding, and integrating ideas. To support this growth, it is important to find 
opportunities for students to take agency over their learning. Self-assessment can provide 
just such an opportunity. Research suggests self-assessment can be an important 
component of the instructional process and plays a central role in scaffolding student 
learning and idea improvement (Chan & van Aalst, 2004; Gipps, 2002; Shepard, 2000). 
Graduate students’ experiences of the academy will influence their scholarly identity 
(Anderson, 1996; Anderson & Louis, 1994; Hodgson & Simoni, 1995; Weidman & Stein, 
2003) and using a process of ongoing reflection as an assessment tool can allow 
students an opportunity to develop a personal voice and perspective towards the material 
and ideas they are studying. As well, assignments like learning journals that sample 
learning over time provide a more thorough picture of change (Chan & van Aalst, 2004). 
Lee, Chan, and van Aalst (2006) have also demonstrated the value of journaling in a 
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knowledge building/Knowledge Forum® context in which students reflect on learning, not 
only through self analysis of their personal change but also analysis of group change 
(also see Niu and Van Aalst in this special issue). 

Assessments are typically framed within belief-mode epistemology with a focus on 
content that is believed or ought to be believed, and with issues of agreement, 
disagreement, and warrant for or against propositions, weight of evidence, arguments 
and counter arguments. Education has historically equipped students to think in belief 
mode and hence, to also test and assess in this mode. Students are expected to examine 
concepts and consider what they and their peers believe, invoking the use of evidence 
and logic to help them take a critical stance toward these beliefs and to evaluate truth 
claims. Design-mode epistemology, by comparison, focuses on idea generation and 
improvement; in particular, the adequacy and improvability of ideas, and whether ideas 
are really leading to a resolution of problems in a field (Bereiter & Scardamalia, 2003). 

Overall, design-mode work is guided by anticipation that a better idea will be forthcoming. 
The goal of design-mode thinking is not consensus but rather “inference to the best 
explanation” (Lipton, 2004). 

While design-mode thinking is essential for advancing beyond what is known, belief-mode 
thinking is required to check the soundness of assumptions, beliefs to be taken into 
account, and the reliability of data. Each mode of thinking thus plays an important role in 
graduate student development. However, design-mode thinking has not traditionally been 
a focus in education at any level (Bereiter & Scardamalia, 2003). In this paper, we 
examine students’ learning journal entries for evidence of epistemological growth in both 
belief and design mode^ Because belief-mode thinking is traditionally part of education, 
particularly higher education, one might find such growth even in the relatively short time 
span of a university course. However, design-mode thinking requires an entirely different 
kind of enculturation and may be harder to observe. One interesting question for this 
paper is therefore whether using learning journals can also encourage design-mode 
thinking, and under what instructional conditions. 

Methods 

Data were collected in three graduate distance education courses taught over two 
consecutive years (n=12, n=15, n=17) with a mixture of Master's and Doctoral students, 
many of whom were taking most of their courses online. In each case the majority of the 
students were at the Master’s level and female. Some lived close to the university but 
preferred the convenience of this online method. Participants in all three courses ranged 
in age from early 20s to mid-50s. 

Procedures 

Each learning journal took the form of a note in the conference to which students added 
weekly from weeks 2-13 during the course. These notes were individually authored but 
part of the shared database and so available to the whole class. The questions were 
designed to encourage weekly reflection on course content and discussion participation. 
The courses were conducted by asynchronous computer conferencing using Web 
Knowledge Forum, a collaborative conferencing system developed at OISE/UT. 
Knowledge Forum differs from typical conferencing systems in that later versions have 
features such as scaffolds, annotations and “rise-above” or summary note capabilities, 
and allow lateral linking and reference to be created between notes in different topic 
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views. 

Design-research methodology (Bereiter, 2002a; Brown, 1992; Collins, 1992) was used, 
with focus on continual improvements within and across the three courses. Changes 
across courses related to the following main issues and are summarized in Table 1: a) 
mandatory use of the learning journal questions; b) explicitness of the connection 
between the learning journal questions and the course content; c) instructor- or self- 
evaluation of online learning journals and d) level of support and scaffolding provided 
during each course to support the journaling process. Each of these changes was 
identified by students as affecting their level of participation and engagement. 

a) Use of journal questions 

In the first course only two of the 18 students spontaneously used the learning journal 
questions until prompted during weeks 7 and 13. In week 7, participants were specifically 
asked to use the questions and this prompted 10 participants to respond. As well, 2 
participants did not use the questions at all during the course, which seriously affected 
the role of their contributions in the analyses. The sporadic use of the questions by 
participants also limited the opportunity to accurately assign student’s entries to 
epistemological levels. Hence, for the second and third course, the use of these 
questions became a course requirement. Additionally, in the second and third courses, 
the learning journals were one of the course assignments and learning journal questions 
were introduced on a weekly basis. 

b) Question content 

For the first course offering, the learning journal questions had been based on all twelve 
of Scardamalia’s (2002) Knowledge Building principles. This proved too ambitious in 
terms of producing measurable effects over a short 13-week course. It was decided to 
concentrate on fewer principles for the second course, specifically those that seemed to 
correlate well with Kuhn’s et al. (2000) and Moshman’s (1998) epistemological 
frameworks, particularly questions related to idea improvement. Additionally, results from 
the first course revealed that the set of reflection questions used were inadequate in 
identifying belief-mode epistemological levels as students focused mostly on content. 
Therefore, for the second course, the questions focused upon the discussion group 
activity for that week and were designed to elicit metacognitive reflection, with efforts to 
tap design-mode thinking, for example, “Reread your theory of learning (the pre-test). 
Pick and discuss one idea that has evolved;’’ and, “Reread your learning journal. How, if 
at all, has this journal helped you? Could it help you more?” In the third course, reflection 
questions were designed to take into consideration the content of the discussions from 
each week, as well as being related to the course theme of increasing progressive 
discourse. 

c) Form of assessment 

At the conclusion of the first course, the learning journals were assessed summatively by 
the course instructor. After discussion with the students however, it became evident that 
this external judgment contradicted the goals of the journals, that of giving the students 
greater sense of voice and ownership in their learning. Accordingly, the journals were 
self-assessed in the second and third offerings of the course. For example, in the second 
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course students were asked to review a rubric and use this as a guide in arriving at a self- 
assessment mark. In this rubric, to self-rate at the highest level a student would need to 
consider that their “Entries were regular and characterized primarily as very reflective 
(i.e., reacting to ideas from the course materials, elaborating ideas from readings and 
from others’ contributions, and contemplating or extending ideas from readings and from 
others’ contributions). As well, entries monitor the learner’s own growth in understanding’’. 
In the third course students were given the rubric as a starting point, with the option for 
students to modify the rubric if desired. Students did not modify the rubric but rather used 
it as initially presented as a frame in which to justify their journal self-rating. 


d) Amount of scaffolding 

In the first two courses the questions themselves constituted the scaffolds for the 
journals. However, for the third course, students were introduced to the course theme of 
progressive discourse (Bereiter, 2002b) and encouraged to use scaffolds in the form of 
Discourse for Inquiry (DPI) cards that were adapted from classroom materials originally 
developed by Woodruff and Brett (1999) to help students take a more advanced 
approach to face-to-face collaborative discussion. The purpose of these DPI cards was to 
scaffold students in developing productive norms and discourse structures for design- 
mode thinking in the online course discussions. One card particularly relevant for the 
focus of this paper was the “managing problem solving” card highlighting Bereiter’s six 
commitments to progressive discourse: a focus on conceptual artifacts; improvability as a 
positive attribute of conceptual artifacts; common understanding given priority over 
agreement; commitment to expand the factual base; selective criticism based on 
knowledge-advancement goals; and nonsectarianism (2002, pp. 87-88). These cards 
were for students to use as reminders of ways to conduct the online discussion. In 
addition, as part of the Knowledge Porum software, there were online scaffolds that 
students used to tag relevant aspects of their contributions. 

Table 1 offers a summary of design research changes to the learning journal assignment 
over three graduate courses. 

Table 1. Design Research Modifications by Course 


Modifications 

Graduate 

Graduate 

Graduate 

in each iteration 

Course I 

Course 2 

Course 3 

Choice 

Choice of whether 

Question use mandatory 

Question use 

to use questions 


mandatory 


Response to one 

Self-evaluation of learning 

Response to one 

Question content 

of a set of pre- 
developed 

journals on a four-level 
rubric 

weekly question 
linked to scaffolds 


questions 


and course content 


Instructor 

Self-evaluation of learning 

Self-evaluation of 

Evaluation 

evaluation of 

journals on a four-level 

learning journals and 

learning journals 

rubric 

option to modify 
rubric as desired 




Suggestions in course 

Training and use of 



materials and videos about 

scaffolds focused 

Scafl'olds 


approaching learning 

explicitly on 


journals 

supporting 




progressive 

discourse 
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Data Sources 

All data came from the learning journal notes within each course database, and the pre- 
and post-essays. Participation and interaction patterns were generated from the data 
captured automatically from the Analytical Toolkit (Burtis, 1998) a quantitative research 
tool developed in conjunction with Web Knowledge Forum at OISE/UT. It extracts such 
summary data as number of notes read and written as well as who read and/or 
referenced others’ notes, and these can be extracted at the individual or at the class 
level. Additionally, learning journal entries were coded using NVivo, for the presence of 
metacognitive activity. A pre- and post-test of a set of questions about students’ 
epistemology of learning was also used to capture changes in levels of epistemological 
understanding as well as how students constructed justifications of that understanding. 
Questions did not point participants directly to the need for continual idea improvement, 
but they provided scope to engage in such efforts, for example, “What do you currently 
understand learning to be — for yourself as a learner and for your students if you teach?” 
and “What role does knowledge play in learning?” 

Results 

In the first part of the results section we present two analyses to examine whether 
learning journals supported deeper understanding (Sanders, 2001) and idea 
improvement, with evidence of work in design-mode. The first analysis compares online 
activity between students who displayed high versus low participation and interactivity as 
measured by the Analytic Toolkit performance data, and the second examines changes in 
metacognitive activity within the learning journals. Level of interactivity (high versus low) 
was assessed through tracking data derived from the Analytic Toolkit. Specifically, high 
participation rates were calculated based on a total per person of the number of notes 
read, written, linked, and revised. These totals were ranked and the classes divided into 
two groups based on that ranking, high and low. 

In the second section we will look at how learning journals can help students make 
connections between new ideas and what they already know (Kerka, 1996), through 
examining descriptions of epistemological changes within the learning journals: these are 
examples where students strive for coherence in their views of knowledge. In the third 
section we identify issues related to student ownership and voice (Kerka, 1996) through 
examining significant issues raised by students within the journals, particularly whether 
journals should be instructor or self-assessed; whether the format should involve 
scaffolded or open formats and the question of privacy. 


1. Do learning journals show evidence of deeper understanding? 

The performance data collected by the Analytic Toolkit gives a quantitative indicator of 
participation and interactivity. In previous research, for example, Chan and van Aalst 
(2003) studied knowledge building portfolios where students identified collective 
knowledge advances showing the community’s best work and progress. They found 
measures of participation (number of notes read, written, linked, revised) correlated with 
portfolio scores and conceptual understanding. In another study, Zhang, Scardamalia, 
Lamon, Messina, and Reeve (2007), identified a strong relationship between the 
presence of extensive writing and reading, and a number of quantitative indicators of 
participation including build-on notes, rise-above notes (summaries and higher-order 
syntheses), referencing, the use of scaffolds with knowledge building, as measured by 
independent analyses of the note content for levels of knowledge advancement. 
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Analysis 1: Online activity 

High scores on participation and interaction in this study inciuded iinking to other’s notes, 
reading iarge amounts of the database, revising own notes and buiiding on other’s ideas. 
Resuits indicated that participants with high overaii participation and interaction rating 
were more iikeiy to use sophisticated justification, such as diaiecticai refiection. 
Additionaiiy, these students were more metacognitive within the iearning journais, were 
more coiiaborative in the course, and were more often rated as Evaluativist in the post- 
essay (the most sophisticated ievei of beiief-mode epistemoiogicai understanding in 
Kuhn’ et ai. (2000) modei). Of particuiar note, students who showed signs of deveiopment 
in their ievei of epistemoiogicai understanding aiso showed increased sophistication in 
justifications from pre to post-essay. For exampie, in answering the question “What roie 
does knowiedge piay in iearning?” one participant starts with a conventionai framing of 
knowiedge in her pre-essay. 

As a teacher, I feel it is important to be knowledgeable about different kinds of 
learning. How do I go about ensuring that I am attempting to meet the needs of all 
learners? I include different kinds of material for different purposes remaining specific 
on the context in my lessons. Appropriate context is important if comprehension and 
learning are to occur. I have to know the strengths and weaknesses of my students so 
that I can build on those strengths and work at improving the areas of concern. I need 
to know the curriculum, understand its philosophical underpinnings, and connect my 
students with information and knowledge that allow them to explore ideas, acquire and 
synthesize information, and solve problems. (Karen, Course 2) 

However, by her post-essay, we see an important shift both in how she is thinking about 
knowiedge and in deriving coherence from severai different sources: 

Knowledge is acquired. It is based on experiences. Experiences are acquired through 
activities which lead to new knowledge, to knowledge building and to further 
understandings (Bereiter, 2002a). Through metacognition in the ‘process of thinking 
about thinking’, new strategies for thinking are developed resulting in ‘deliberate, 
planful, and goal-directed thinking applied to one’s thoughts to accomplish cognitive 
tasks (Hacker). Knowledge building as viewed by Bereiter most appeals to my sense 
of what ‘teaching’ of knowledge entails and the role that it plays in learning as it 
accompanies knowledge building. Ideally, knowledge building contributes to new 
knowledge in learning through process, through collaboration, through reasoning, and 
to ever deeper processing of information. I find it noteworthy to include that John 
Dewey spoke, some 100 years ago, in his work, to the idea of pedagogical evolution in 
leading to knowledge building. (Karen, Course 2) 

From a design-mode perspective, there was some indication of shifts in conceptuai 
understanding among students in the first and second courses that couid be 
characterized as idea improvement, but often the improvement was presented in the form 
of a retrospective anaiysis - a recounting of what was iearned and a statement of 
intention to chart a new iearning direction or a new educationai mission. However, we did 
not detect work in design mode with our scoring scheme (e.g., students identifying the 
iimits of an idea to accompiish some needed change, addressing design constraints in an 
effort to define a better way, and so forth). The scoring scheme refiects the difference 
between eiaborating beiiefs (for exampie, beiiefs about the importance and usefuiness of 
knowiedge buiiding) versus improving an idea (how knowiedge buiiding itseif can be 
improved). More generaiiy, resuits suggest participants did not see themseives as agents 
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of educational change. Their goals were mostly focused on reconciling different views of 
given information — a belief-mode advance — rather than developing new knowledge, a 
marker of design mode and knowledge building. Thus while there was considerable use 
of metacognition and advances in use of sophisticated justification, there was little 
evidence of design-mode epistemology. 

Analysis 2: Shifts in metacognitive activity over time 

Typically, across all courses, students who were metacognitive were also rated high in 
participation and were metacognitive with or without the questions. A student reflects. 

In discussing metacognitive experiences I reaiized that this iearning journai 
encouraged metacognitive activity. The thinking about iearning about thinking about 
teaming.... is heiping me team about thinking about iearning :-) The journai exercise 
has aiso ied me to some goat setting for the week to come. (Sue, Course 2) 

In contrast, students who seemed less self-reflective about their learning (many fewer 
metacognitive examples were coded for these students) showed greater metacognition 
when using the scaffolded questions that appeared to support their metacognitive 
reflection, a position supported by comments such as the following: 

/ do enjoy using the iearning journai as a piace to store my thoughts about the week 
and to encourage refiection. The questions posted heip me to move in directions that I 
might not have thought of. Without the questions, I wouid probabiy tend to do more of 
a summary of the readings rather than use it for appiication and understanding. 
(Katherine, Course 2) 

Students rated as having a high activity level also tended to ask themselves 
metacognitive questions when designing their own learning journal questions later in the 
course. By contrast students who were rated low in participation and interaction tended to 
display metacognitive activity only when prompted. Only one of the low activity rated 
students asked themselves reflective questions during those last few journal entries. A 
student rated as having low relative activity within the second course said. 

My first experience with these kinds of refiective journais for assessment was in the 
previous course I took with [instructor]. I must admit they are hard to get used to and, 
aithough they are a great tooi to engage in metacognition, they are hard for me to do 
on a reguiar basis. I know many others in the course iove working on them and find 
them very usefui, but my feeiings are a iittie more mixed. (Laurent, Course 2) 

However for another student who received very high relative activity ratings, the purpose 
and value seemed crystal clear, 

/ fee/ that I did a iot of metacognitive thinking about my own iearning and this was one 
of the main points of writing a iearning journai (wasn’t it?) - to monitor our own 
progress and to cieariy identify what issues we couid dear up and which ones we 
couidn’t. Aiso to “rise above” by trying to integrate our iearning from various weeks. 
These are my impressions of why we shouid be writing iearning journais. (Jane, 
Course 2) 

From a knowledge building perspective we do not see attention to community knowledge 
and collective responsibility but rather to self-analysis of personal learning (more in 
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keeping with rationalist than knowledge building epistemology, as indicated above). 


2. Did learning journals help students make connections between new ideas and 
what they already know? 

A tangible benefit of the learning journals was the degree to which they supported 
dialectical reflection consistent with the rational constructivist thinking believed to be 
necessary at the graduate school level. In the examples below, we can see the use of 
dialectical reflection to highlight changes in epistemological views. Almost 100% of 
participants made such comments at some point during their course. With respect to the 
relative nature of knowledge, a participant discovers that. 

Sue didn’t need what I did, she did just fine with gray. I think there are advantages in 
both ways of thinking but my point is that I didn’t reaiize untii now that I can oniy think 
in biack and white. So I have some teaming to do. (Manon, Course 2) 

From a design-mode thinking perspective, this student recognizes the need for learning, 
but is not identifying ideas to be improved. There is, for example, no elaboration of an 
idea that might extend beyond black-and-white analysis. Another participant recognized 
that knowledge was not finite: 

/ think one of the biggest personai reveiations during the course so far is that there is 
no “end point” so to speak regarding knowiedge. It is aiways growing, changing, 
evoiving, and aid concepts are continuaiiy revisited and tested. It is truiy dynamic, and 
I think that’s why the pursuit of knowiedge is so exciting. (Ruth, Course 2) 

Again, from a design-mode perspective Ruth recognizes knowledge as involving 
continual change, but there is no suggestion that the author sees herself as an agent in 
that change. Change is, in this sense, a “primitive” - something that occurs without 
presumed need of explanation. Ohisson (1991) noted that this is how students who lack 
an understanding of evolution treat the concept - they report that it occurs and that it is 
important, but indicate no need to understand or explain it further. 

The next participant came to realize the socially constructed nature of knowledge and 
hence the importance of others to her learning. 

This week I began to deveiop more of an appreciation for the sociai aspects of 
teaming. This occurred on a personai as weii as theoreticai /eve/. / began to see the 
impact of society and the environment on teaming in “generai” and atso began to 
appreciate the effects of these factors on my own teaming. I reatized that what I wouid 
take away from this course/experience was directty rotated to the others in the ctass, 
my interactions with them and their ontine contributions. As someone who generatty 
saw my “textbook” as my greatest resource, this was a big teapi (Sue, Course 2) 

The student suggests a shift from the “textbook” as her absolute “greatest” resource for 
learning and from a “repository of knowledge” to some “big leap.” But what was that big 
leap and in what sense would it represent an improvement over her earlier ideas? From a 
design-mode perspective, there is no attempt to deal with the challenges of textbook 
learning and how one might actually improve upon it. Yet the textbook itself can be seen 
as a social artifact and hence she was already at some level “socially interacting”, 
something she recognizes later in the course: 
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Previously, I hadn’t recognized that the artifacts which I used, even while working 
independently, were representations of distributed intelligence . . . therefore linking me 
with this web and rendering my independence merely relative and, ultimately, an 
illusion. I have begun to recognize the subtle and often invisible contributions of others 
to my so called independent work and learning. (Sue, Course 2) 

In the third course we see other kinds of connections being made between existing and 
new knowledge. The following excerpts are from a student in the third course, showing 
important changes of the sort made by other students. Her first comment illustrates the 
understanding of knowledge as something to be shared rather than just individually 
constructed 

At this point in civilization has all the knowledge already been created by someone at 
some point of time. Are we now just realizing that we have to somehow pull all that 
knowledge together, is the internet Just the tool to do that for us? ... my personal goals 
for this course are to get a greater understanding of what knowledge is and how it can 
be shared. I would also like to improve my writing skills and learn to work better in a 
group. (Chloe, Course 3). 

Her next excerpt recognition that knowledge building is work with ideas that can be 
improved over time: 

Yes I think my personal learning goals for this course have changed. I have doing a lot 
of work with Adam and learning quite a bit from him. We have had many discussions 
or should I say dialogs having just read Belinda’s note. Through these discussions I 
have come to realize that most if not all of my contributions have been opinions and 
not ideas and as such they have not contributed to knowledge building. (Chloe, 
Course 3) 

This insight seems to then impact her behaviour in the course and she begins to display 
agency by taking action to engage in more progressive discourse/design-mode activity. 
She starts to address entries towards the whole class rather than to a particular person 
and also begins to explicitly use the Discourse for Inquiry cards to help her engage more 
collaboratively. As she explains here: 

Other changes that I have to make to my contributions are not so small or easy for me 
at least to make. I have the “Discourse for Inquiry” cards in front of me at all times and 
I am really trying to incorporate them into my contributions but I fear I am falling short. I 
asked Adam if he used them during one of our discussions and he said no as the 
concepts are so ingrained into him that he doesn’t need them any longer. I can only 
hope that one day I can say the same thing but right now I need them and would be 
happy even to get to the point where I can say that I am contributing to knowledge 
building and the advancement of knowledge instead of just knowledge sharing. (Chloe, 
Course 3) 

3. Did learning journals provide a context for student ownership and development 
of student voice? 

The evidence for the journals providing a location for student ownership and the 
development of student voice came from their reflections on three particular issues. The 
first was whether the learning journals should be instructor or self-assessed; the second 
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was the issue of whether scaffolded or open-format questions should be provided; and 
the third was whether journals should be private or public. 


Instructor Assessment vs. Self-Assessment 

For the first course, learning journals were instructor assessed. In course feedback 
students pointed out that in spite of discussion about constructivism and the focus on 
student agency for learning, grading was done exclusively by faculty. As this seemed 
contradictory to the goal of graduate student agency, a self-assessment component in the 
form of the learning journal assignment was added for the second course. Students were 
required to send an e-mail to the instructor at course end with a self-assigned grade 
related to a small rubric along with a brief justification. In addition to the actual grading of 
the overall journals, many of the questions also encouraged students to self-assess. We 
believe that this is a valuable skill is required by all students who will be lifelong learners. 
It has also been associated with the higher levels of epistemological development (Baxter 
Magolda, 1999; Perry, 1970). One student shows her astonishment concerning the self- 
assessed nature of the learning journals. 

We even have a chance at self-assessment in this course through our Learning 
Journals... what amazing power in the hands of students I What a change from a 
traditional classroom. (Sue, Course 2) 

Another student expresses the value of metacognitive reflection afforded by the journals 
cogently. 

The learning journal is a good knowledge building device/exercise that tracks 
reflections and a good way to assess my learning and theoretical growth to date. The 
inclusion of this assessment tool has forced me to sit down and record my thoughts, 
which I believe to be paramount for complex individual learning to occur. The journal is 
a good record of discoveries and a place to return to during and after the course to 
reflect on what I said, I have learned, and confirm the validity of my reflections as the 
course progresses and thereafter. (Bev, Course 2) 

For some students, the journals allowed them to examine aspects of their own thinking 
that were challenging, and in the third course, only two of 17 students, each of whom had 
low participation levels, expressed difficulty with the process of self-assessment: 

/ have always found self-evaluation difficult. I tend to be a perfectionist and I really do 
find looking at myself difficult. I often seek right/wrong or black/white answers, so I find 
reflection difficult, I can’t tell if I am “right” in a reflection. Also, it is difficult to assess, 
should I be looking at what I perceived or what I actually did? j(Anne, Course 3 

Thus while self-assessment can be helpful, there is considerable variability in how 
comfortable students are with that process. 

Scaffolded or Open Format 

There was much discussion about whether the journals should be scaffolded using 
questions or just left open to the student’s own decisions about content. A student who 
showed high participation in online discussion reflects on the advantages of having no 
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preset questions, 

In rereading this journal, I found that the content was more valid for me when I took an 
open-reflection approach. As I reread the journai, however, I found that this open 
approach tended to be more content driven, and not as personai, or refiective as when 
I took a more hoiistic approach. Despite the benefits of content recap and synopsis, I 
did notice a iack of introspection with this approach. (Mike, course 2) 

He then reflects on having questions provided. 

The guidance that was provided throughout this course in our iearning Journais 
concentrated primariiy on “refiection”, and as such, has huge benefits that I wouid not 
have otherwise reaped. When I took this introspective approach, however, I did miss 
the content-synopsis that wouid come with a more open and free fiowing approach to 
this Journai. (Mike, Course 2) 

One who participated very little in the discussions said 

/ do enjoy using the iearning journai as a piace to store my thoughts about the week 
and to encourage refiection. The questions posted heip me to move in directions that I 
might not have thought of Without the questions, I wouid probabiy tend to do more of 
a summary of the readings rather than use it for appiication and understanding. 
(Manon, Course 2) 

In the third course, a student with high participation levels who wrote entries before 
reflection questions were begun in weeks noted: 

The refiective journai was something that I thought couid be tweaked a iittie. I found 
eariier on that my entries were not in-depth and personai enough, so I struggied with 
them. After [instructor] changed them and gave us the theme/questions, this heiped 
me remain focused and on track. (Jeff, Course 3) 

This student was the only one in the third course to comment on the issue of scaffolded 
as opposed to open format questions, likely because it was a consistent strategy in place 
from the beginning of the course. 

Overall, it appeared that having questions to respond to is most useful either to orient 
students new to the journaling process, or as a scaffold for those who might otherwise be 
less metacognitive. It did appear as if questions were occasionally actually demotivating 
to the most independent and active students and while, for research purposes it was 
helpful to have everyone respond to similar questions, instructionally, it might be better to 
scaffold using questions initially, but relax the requirement as the course proceeds. 


Private or public and the nature of journals 

Perhaps the most contentious issue was whether the journal should be public for all 
students in the course to see or private to be shared just with the instructor and TA. This 
issue is also reflected in other research on journaling (e.g., Hansen, 2005; Kerka, 1996; 
Wickstrom, 2003). Interestingly, while one of the cited problems of public writing is that it 
may change what students will say because of the audience, this may not necessarily be 
negative, as the public nature of the online journals allows modeling of interesting styles 
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of journaling; the opportunity for peer feedback on entries and a deeper understanding of 
different ways others are understanding the ideas in the course. Whether the public 
element works positively would seem to be largely dependent upon the learning 
environment and collaborative tone set for the course by the instructor. In the present 
study there seemed to be a fairly evident split between students who had high versus low 
relative activity as to how they felt about other’s having access to their journals. Students 
with high levels of activity were eager to share their journals and seemed to learn much 
from reading others journals. One student states, 

Reading other people’s journals has also been a learning experience. Even though we 
all read the same articles, answer the same questions and use the same “folders”.. .our 
experiences are so unique and varied. (Jane, Course 2) 

By contrast, students who had a relatively low activity rating typically preferred more 
privacy. For instance, one student said. 

An issue that I’ve faced with the format for the journals is that they are public; I’d prefer 
them to be private or maybe just shared with [the instructor] and to have the option of 
posting to a public forum if one wishes. (Katherine, Course 2) 

Another student framed her reluctance to make her journal public another way 

If I could choose, I would prefer having my learning journal not published. Since it’s 
named as “journal”, I have difficulties sharing my journal with others. Therefore, it 
always took me a long time to think of what I could write. I just could not write down 
everything that came to me easily when thinking of the fact that they would be read by 
others. (Christine, Course 3) 

The second of these entries suggests that journals are by nature private, and it may be 
that ultimately, comfort with a public venue may hinge on each student’s understanding of 
the learning journal purpose, an understanding that is influenced by prior experience. In 
course 3, six out of 17 students preferred journals to be private, though two of these 
students conceded the usefulness of public journals, especially at the beginning of the 
course, and enjoyed being able to read others’ journals. One student explicitly stated that 
she did not mind others reading her journal, but felt uncomfortable commenting on others’ 
journals. 

Additionally, in the third course, two students (with high activity ratings) had prior 
experiences with online journaling in other courses by the same instructor. One student 
already kept a personal weblog to which she posted regular entries. Two other students 
voluntarily chose to weblog making their journals public to a broader audience. Another 
student in this course stated how the public nature of the journals allowed her own 
conception of the purpose of journals to change: 

/ know I was using it more as a journal when I started, but after reading Belinda’s log, 
where I could see her working through various concepts, the light came on. Ah, this is 
what this journal is for. So, I attempted to do the same thing. It was an inner 
monologue or debate - looking at issues from different perspectives and trying to come 
to some sort of conclusion. (Megan, Course 3) 

It seems clear that those students who arrive in online classes already using personal 
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Weblogs find the public nature of journals much less controversial. It appears to be those 
students who are unused to such forms of public discourse for whom it is most 
challenging. For these students a clear explication from the beginning of the course of 
differences between personal and public in relation to course ideas and understanding 
may well be of assistance in making them feel more comfortable. 

Conclusions and Educational importance 

Much success has been reported about the adoption of online journaling (Admiraal et al., 
1999; Andrusyszyn & Davie, 1997; Graybeal, 1987), and more recently, online weblogs 
show potential similarities and an alternative technical structure for such journaling. In our 
research, students’ learning journals were accessible to all within the conference. While 
not enjoyed by all students, those with higher activity levels preferred this method. 
Further, it was clear that the learning journals encouraged explicit recording of students’ 
conceptual and metacognitive growth. While there was not much discussion by students 
concerning the self-assessment of their learning journals within the journals themselves, 
other communications with the instructor (mainly e-mail) suggested to the instructor that 
the range of self assigned grades was linked to their actual reflective performance in a 
way that was probably more accurate, and more useful from a learning perspective than if 
they had been instructor-assessed. Student commentaries indicate that the purpose of 
the journal needs to be well publicized, and explained in a variety of ways to help 
students become comfortable with the process. Further, while not required by some 
students, questions that scaffold metacognition should be available. 

Instructionally, online journals have a number of advantages. First, they are inherently 
flexible and can be integrated anywhere in a course. Second, they are also open-ended 
and can be designed with whatever degree of structure suits the particular instructional 
context in which they are being used. Third, they can be framed to both direct and assess 
current understanding using common questions that students respond to at particular 
times. Fourth, they can easily be instructor-assessed or student-assessed by 
collaboratively developing a rubric or other criteria to apply to the journals. Finally, 
learning journals support student voice and ownership by encouraging the integration of 
course ideas with student’s existing understanding through a process of sustained 
reflection. With such explicit support they can also provide a location for both students 
and instructors to monitor design-mode thinking. 

Evidence of work in design mode was all-but-absent in journal entries. This suggests that 
students see themselves as learners of educational innovations rather than designers of 
next-generation educational methods and environments. Follow-up research will be 
required to determine whether making design mode more explicit and engaging students 
directly in collective knowledge construction in design mode may lead to greater 
awareness of knowledge building epistemology and educational innovation based on it. 
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